Systems and methods for instrumentation of education processes

ABSTRACT

Systems and methods for education instrumentation can include one or more servers configured generate a plurality of models for modeling various aspects of an education process using training data related to academic performance of students. The one or more servers can collect data from client devices associated with various education institutions or stakeholders throughout a life cycle of the education process. The one or more servers can use the generated models and the collected data to assess the addressing of education standards and predict or estimate performance metrics associated with the education process. The one or more servers can provide computed metrics or assessments of how well education standards are addressed to one or more client devices for display.

RELATED APPLICATION

This application claims the benefit under 35 U.S.C. § 119(e) of U.S.Provisional Application No. 62/250,933, entitled “SYSTEMS AND METHODSFOR MODELING EDUCATION DATA AND PREDICTING ACADEMIC PERFORMANCE” andfiled Nov. 4, 2015, which is incorporated herein by reference in itsentirety.

FIELD OF THE DISCLOSURE

The present application relates generally to systems and methods for theinstrumentation of the education process. Specifically, the presentapplication relates to computerized methods and systems for determiningthe relative importance of, and the dependency relationships between,various factors that influence education processes, and predictinglearning outcome.

BACKGROUND

Most countries aspire and struggle to improve and fix their educationsystems and so do many districts and schools. In the effort forcontinuous improvement in education, schools, school districts,colleges, universities, professional development institutes, departmentsof education, organizations or private sector companies may conductvarious types of analyses that may be qualitative or quantitative innature. Such analyses are usually based on academic assessment scores.

SUMMARY

According to one aspect, a system for computing performance metrics ofeducation processes the system can include one or more computer serversand one or more client applications running on a plurality of clientdevices. The one or more client applications can provide a plurality ofuser interfaces associated with a plurality of stages of a life cycle ofan education process. The one or more servers having a database, a datacollector, an analysis module, a simulator, and an output module. Thedatabase can store training data and test data related to studentacademic performance of a first plurality of students. The datacollector can receive student information data for a second plurality ofstudents from a student information system (SIS). The data collector canreceive, from one or more client devices, one or more sets of standardelements for one or more lesson plans selected via a first userinterface associated with the one or more client devices. The first userinterface can include lesson planning templates for selection ofstandard elements for lesson planning. The analysis module can generate,using the training data, a plurality of models to simulate performanceof an education process, and validate the plurality of models using thetest data. The simulator can select a model from the plurality of modelsbased on a selection by a client device of a student population amongthe second plurality of students. The simulator can compute aperformance metric using the selected model and at least a subset of thestudent information data and the one or more sets of standard elements.The output module can provide the computed performance metric fordisplay on the plurality of client devices via a second user interfaceof the plurality of user interfaces.

According to one other aspect, a method for computing performancemetrics of education processes can include storing, by one or moreprocessors, training data and test data in a database. The training dataand the test data can be related to student academic performance of afirst plurality of students. The one or more processors can receivestudent information data for a second plurality of students from astudent information system (SIS), and receive, from one or more clientdevices, one or more sets of standard elements for one or more lessonplans selected via a first user interface associated with the one ormore client devices. The first user interface including lesson planningtemplates for selection of standard elements for lesson planning. Theone or more processors can generate, using the training data, aplurality of models to simulate performance of an education process, andvalidate the plurality of models using the test data. The one or moreprocessors can select a model from the plurality of models based on aselection by a client device of a student population among the secondplurality of students. The one or more processors can compute aperformance metric using the selected model and at least a subset of thestudent information data and the one or more sets of standard elements.The one or more processors can provide the computed performance metricfor display on the client device via a second user interface of theplurality of user interfaces.

According to another aspect, a computer-readable medium can includecomputer code instructions stored thereon, which when executed by one ormore processors of a first device, cause the one or more processors tostore training data and test data in a database. The training data andthe test data can be related to student academic performance of a firstplurality of students. The one or more processors can receive studentinformation data for a second plurality of students from a studentinformation system (SIS), and receive, from one or more client devices,one or more sets of standard elements for one or more lesson plansselected via a first user interface associated with the one or moreclient devices. The first user interface including lesson planningtemplates for selection of standard elements for lesson planning. Theone or more processors can generate, using the raining data, a pluralityof models to simulate performance of an education process, and validatethe plurality of models using the test data. The one or more processorscan select a model from the plurality of models based on a selection bya client device of a student population among the second plurality ofstudents. The one or more processors can compute a performance metricusing the selected model and at least a subset of the studentinformation data and the one or more sets of standard elements. The oneor more processors can provide the computed performance metric fordisplay on the client device via a second user interface of theplurality of user interfaces.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram depicting an embodiment of a networkenvironment comprising local devices in communication with remotedevices.

FIGS. 1B-1D are block diagrams depicting embodiments of computers usefulin connection with the methods and systems described herein.

FIGS. 2A and 2B show charts illustrating, respectively, the averagespending per student and the numeracy proficiency scores for a pluralityof countries.

FIG. 3 shows a diagram illustrating an education instrumentation (EI)computer platform, according to one or more embodiments.

FIG. 4 shows a block diagram illustrating an architecture of an analysismodule illustrated in FIG. 3, according to one or more embodiments.

FIG. 5 shows a chart illustrating the strength or dominance of aplurality of variables with regard to predicting student learningoutcome in science for Germany in 2009.

FIG. 6 shows a Bayesian network model generated by the EI system in FIG.3, according to one or more embodiments.

FIG. 7 shows a diagram illustrating various stages in a life cycle of aneducation process.

FIGS. 8A-E show various UIs associated with a curriculum planning stageof the life cycle shown in FIG. 7.

FIG. 9 shows a UI associated with the instruction stage of the lifecycle in FIG. 7.

FIGS. 10A-C show various UIs associated with the reporting stage of thelife cycle in

FIG. 7.

FIG. 11 shows a flowchart of a method for computing performance metricsof education processes.

DETAILED DESCRIPTION

For purposes of reading the description of the various embodimentsbelow, the following descriptions of the sections of the specificationand their respective contents may be helpful:

Section A describes a computing and network environment which may beuseful for practicing embodiments described herein.

Section B describes a computer environment for instrumentation ofeducation processes.

A. Computing and Network Environment

In addition to discussing specific embodiments of the present solution,it may be helpful to describe aspects of the operating environment aswell as associated system components (e.g., hardware elements) inconnection with the methods and systems described herein. Referring toFIG. 1A, an embodiment of a computing and network environment 10 isdepicted. In brief overview, the computing and network environmentincludes one or more clients 102 a-102 n (also generally referred to aslocal machine(s) 102, client(s) 102, client node(s) 102, clientmachine(s) 102, client computer(s) 102, client device(s) 102,endpoint(s) 102, or endpoint node(s) 102) in communication with one ormore servers 106 a-106 n (also generally referred to as server(s) 106,node 106, or remote machine(s) 106) via one or more networks 104. Insome embodiments, a client 102 has the capacity to function as both aclient node seeking access to resources provided by a server and as aserver providing access to hosted resources for other clients 102 a-102n.

Although FIG. 1A shows a network 104 between the clients 102 and theservers 106, the clients 102 and the servers 106 may be on the samenetwork 104. In some embodiments, there are multiple networks 104between the clients 102 and the servers 106. In one of theseembodiments, a network 104′ (not shown) may be a private network and anetwork 104 may be a public network. In another of these embodiments, anetwork 104 may be a private network and a network 104′ a publicnetwork. In still another of these embodiments, networks 104 and 104′may both be private networks.

The network 104 may be connected via wired or wireless links. Wiredlinks may include Digital Subscriber Line (DSL), coaxial cable lines, oroptical fiber lines. The wireless links may include BLUETOOTH, Wi-Fi,Worldwide Interoperability for Microwave Access (WiMAX), an infraredchannel or satellite band. The wireless links may also include anycellular network standards used to communicate among mobile devices,including standards that qualify as 1G, 2G, 3G, or 4G. The networkstandards may qualify as one or more generation of mobiletelecommunication standards by fulfilling a specification or standardssuch as the specifications maintained by International TelecommunicationUnion. The 3G standards, for example, may correspond to theInternational Mobile Telecommunications-2000 (IMT-2000) specification,and the 1G standards may correspond to the International MobileTelecommunications Advanced (IMT-Advanced) specification. Examples ofcellular network standards include AMPS, GSM, GPRS, UMTS, LTE, LTEAdvanced, Mobile WiMAX, and WiMAX-Advanced. Cellular network standardsmay use various channel access methods e.g. FDMA, TDMA, CDMA, or SDMA.In some embodiments, different types of data may be transmitted viadifferent links and standards. In other embodiments, the same types ofdata may be transmitted via different links and standards.

The network 104 may be any type and/or form of network. The geographicalscope of the network 104 may vary widely and the network 104 can be abody area network (BAN), a personal area network (PAN), a local-areanetwork (LAN), e.g. Intranet, a metropolitan area network (MAN), a widearea network (WAN), or the Internet. The topology of the network 104 maybe of any form and may include, e.g., any of the following:point-to-point, bus, star, ring, mesh, or tree. The network 104 may bean overlay network which is virtual and sits on top of one or morelayers of other networks 104′. The network 104 may be of any suchnetwork topology as known to those ordinarily skilled in the art capableof supporting the operations described herein. The network 104 mayutilize different techniques and layers or stacks of protocols,including, e.g., the Ethernet protocol, the internet protocol suite(TCP/IP), the ATM (Asynchronous Transfer Mode) technique, the SONET(Synchronous Optical Networking) protocol, or the SDH (SynchronousDigital Hierarchy) protocol. The TCP/IP internet protocol suite mayinclude application layer, transport layer, internet layer (including,e.g., IPv6), or the link layer. The network 104 may be a type of abroadcast network, a telecommunications network, a data communicationnetwork, or a computer network.

In some embodiments, the computing and network environment 10 mayinclude multiple, logically-grouped servers 106. In one of theseembodiments, the logical group of servers may be referred to as a serverfarm 38 or a machine farm 38. In another of these embodiments, theservers 106 may be geographically dispersed. In other embodiments, amachine farm 38 may be administered as a single entity. In still otherembodiments, the machine farm 38 includes a plurality of machine farms38. The servers 106 within each machine farm 38 can be heterogeneous—oneor more of the servers 106 or machines 106 can operate according to onetype of operating system platform (e.g., WINDOWS 8 or 10, manufacturedby Microsoft Corp. of Redmond, Wash.), while one or more of the otherservers 106 can operate on according to another type of operating systemplatform (e.g., Unix, Linux, or Mac OS X).

In one embodiment, servers 106 in the machine farm 38 may be stored inhigh-density rack systems, along with associated storage systems, andlocated in an enterprise data center. In this embodiment, consolidatingthe servers 106 in this way may improve system manageability, datasecurity, the physical security of the system, and system performance bylocating servers 106 and high performance storage systems on localizedhigh performance networks. Centralizing the servers 106 and storagesystems and coupling them with advanced system management tools allowsmore efficient use of server resources.

The servers 106 of each machine farm 38 do not need to be physicallyproximate to another server 106 in the same machine farm 38. Thus, thegroup of servers 106 logically grouped as a machine farm 38 may beinterconnected using a wide-area network (WAN) connection or ametropolitan-area network (MAN) connection. For example, a machine farm38 may include servers 106 physically located in different continents ordifferent regions of a continent, country, state, city, campus, or room.Data transmission speeds between servers 106 in the machine farm 38 canbe increased if the servers 106 are connected using a local-area network(LAN) connection or some form of direct connection. Additionally, aheterogeneous machine farm 38 may include one or more servers 106operating according to a type of operating system, while one or moreother servers 106 execute one or more types of hypervisors rather thanoperating systems. In these embodiments, hypervisors may be used toemulate virtual hardware, partition physical hardware, virtualizephysical hardware, and execute virtual machines that provide access tocomputing environments, allowing multiple operating systems to runconcurrently on a host computer. Native hypervisors may run directly onthe host computer. Hypervisors may include VMware ESX/ESXi, manufacturedby VMWare, Inc., of Palo Alto, Calif.; the Xen hypervisor, an opensource product whose development is overseen by Citrix Systems, Inc.;the HYPER-V hypervisors provided by Microsoft or others. Hostedhypervisors may run within an operating system on a second softwarelevel. Examples of hosted hypervisors may include VMware Workstation andVIRTUALBOX.

Management of the machine farm 38 may be de-centralized. For example,one or more servers 106 may comprise components, subsystems and modulesto support one or more management services for the machine farm 38. Inone of these embodiments, one or more servers 106 provide functionalityfor management of dynamic data, including techniques for handlingfailover, data replication, and increasing the robustness of the machinefarm 38. Each server 106 may communicate with a persistent store and, insome embodiments, with a dynamic store.

Server 106 may be a file server, application server, web server, proxyserver, appliance, network appliance, gateway, gateway server,virtualization server, deployment server, SSL VPN server, firewall,Internet of Things (IoT) controller. In one embodiment, the server 106may be referred to as a remote machine or a node. In another embodiment,a plurality of nodes 290 may be in the path between any twocommunicating servers.

Referring to FIG. 1B, a cloud computing environment is depicted. Thecloud computing environment can be part of the computing and networkenvironment 10. A cloud computing environment may provide client 102with one or more resources provided by the computing and networkenvironment 10. The cloud computing environment may include one or moreclients 102 a-102 n, in communication with the cloud 108 over one ormore networks 104. Clients 102 may include, e.g., thick clients, thinclients, and zero clients. A thick client may provide at least somefunctionality even when disconnected from the cloud 108 or servers 106.A thin client or a zero client may depend on the connection to the cloud108 or server 106 to provide functionality. A zero client may depend onthe cloud 108 or other networks 104 or servers 106 to retrieve operatingsystem data for the client device. The cloud 108 may include back endplatforms, e.g., servers 106, storage, server farms or data centers.

The cloud 108 may be public, private, or hybrid. Public clouds mayinclude public servers 106 that are maintained by third parties to theclients 102 or the owners of the clients. The servers 106 may be locatedoff-site in remote geographical locations as disclosed above orotherwise. Public clouds may be connected to the servers 106 over apublic network. Private clouds may include private servers 106 that arephysically maintained by clients 102 or owners of clients. Privateclouds may be connected to the servers 106 over a private network 104.Hybrid clouds 108 may include both the private and public networks 104and servers 106.

The cloud 108 may also include a cloud based delivery, e.g. Software asa Service (SaaS) 110, Platform as a Service (PaaS) 112, andInfrastructure as a Service (IaaS) 114. IaaS may refer to a user rentingthe use of infrastructure resources that are needed during a specifiedtime period. IaaS providers may offer storage, networking, servers orvirtualization resources from large pools, allowing the users to quicklyscale up by accessing more resources as needed. Examples of IaaS includeAMAZON WEB SERVICES provided by Amazon.com, Inc., of Seattle, Wash.,RACKSPACE CLOUD provided by Rackspace US, Inc., of San Antonio, Tex.,Google Compute Engine provided by Google Inc. of Mountain View, Calif.,or RIGHTSCALE provided by RightScale, Inc., of Santa Barbara, Calif.PaaS providers may offer functionality provided by IaaS, including,e.g., storage, networking, servers or virtualization, as well asadditional resources such as, e.g., the operating system, middleware, orruntime resources. Examples of PaaS include WINDOWS AZURE provided byMicrosoft Corporation of Redmond, Wash., Google App Engine provided byGoogle Inc., and HEROKU provided by Heroku, Inc. of San Francisco,Calif. SaaS providers may offer the resources that PaaS provides,including storage, networking, servers, virtualization, operatingsystem, middleware, or runtime resources. In some embodiments, SaaSproviders may offer additional resources including, e.g., data andapplication resources. Examples of SaaS include GOOGLE APPS provided byGoogle Inc., SALESFORCE provided by Salesforce.com Inc. of SanFrancisco, Calif., or OFFICE 365 provided by Microsoft Corporation.Examples of SaaS may also include data storage providers, e.g. DROPBOXprovided by Dropbox, Inc. of San Francisco, Calif., Microsoft SKYDRIVEprovided by Microsoft Corporation, Google Drive provided by Google Inc.,or Apple ICLOUD provided by Apple Inc. of Cupertino, Calif.

Clients 102 may access IaaS resources with one or more IaaS standards,including, e.g., Amazon Elastic Compute Cloud (EC2), Open CloudComputing Interface (OCCI), Cloud Infrastructure Management Interface(CIMI), or OpenStack standards. Some IaaS standards may allow clientsaccess to resources over HTTP, and may use Representational StateTransfer (REST) protocol or Simple Object Access Protocol (SOAP).Clients 102 may access PaaS resources with different PaaS interfaces.Some PaaS interfaces use HTTP packages, standard Java APIs, JavaMailAPI, Java Data Objects (JDO), Java Persistence API (JPA), Python APIs,web integration APIs for different programming languages including,e.g., Rack for Ruby, WSGI for Python, or PSGI for Perl, or other APIsthat may be built on REST, HTTP, XML, or other protocols. Clients 102may access SaaS resources through the use of web-based user interfaces,provided by a web browser (e.g. GOOGLE CHROME, Microsoft INTERNETEXPLORER, or Mozilla Firefox provided by Mozilla Foundation of MountainView, Calif.). Clients 102 may also access SaaS resources throughsmartphone or tablet applications, including, for example, SalesforceSales Cloud, or Google Drive app. Clients 102 may also access SaaSresources through the client operating system, including, e.g., Windowsfile system for DROPB OX.

In some embodiments, access to IaaS, PaaS, or SaaS resources may beauthenticated. For example, a server or authentication server mayauthenticate a user via security certificates, HTTPS, or API keys. APIkeys may include various encryption standards such as, e.g., AdvancedEncryption Standard (AES). Data resources may be sent over TransportLayer Security (TLS) or Secure Sockets Layer (SSL).

The client 102 and server 106 may be deployed as and/or executed on anytype and form of computing device, e.g. a computer, network device orappliance capable of communicating on any type and form of network andperforming the operations described herein. FIGS. 1C and 1D depict blockdiagrams of a computing device 100 useful for practicing an embodimentof the client 102 or a server 106. As shown in FIGS. 1C and 1D, eachcomputing device 100 includes a central processing unit 121, and a mainmemory unit 122. As shown in FIG. 1C, a computing device 100 may includea storage device 128, an installation device 116, a network interface118, an I/O controller 123, display devices 124 a-124 n, a keyboard 126and a pointing device 127, e.g. a mouse. The storage device 128 mayinclude, without limitation, an operating system, software, and asoftware of an education instrumentation (EI) system 120. As shown inFIG. 1D, each computing device 100 may also include additional optionalelements, e.g. a memory port 103, a bridge 170, one or more input/outputdevices 130 a-130 n (generally referred to using reference numeral 130),and a cache memory 140 in communication with the central processing unit121.

The central processing unit 121 is any logic circuitry that responds toand processes instructions fetched from the main memory unit 122. Inmany embodiments, the central processing unit 121 is provided by amicroprocessor unit, e.g.: those manufactured by Intel Corporation ofMountain View, Calif.; those manufactured by Motorola Corporation ofSchaumburg, Ill.; the ARM processor and TEGRA system on a chip (SoC)manufactured by Nvidia of Santa Clara, Calif.; the POWER7 processor,those manufactured by International Business Machines of White Plains,N.Y.; or those manufactured by Advanced Micro Devices of Sunnyvale,Calif. The computing device 100 may be based on any of these processors,or any other processor capable of operating as described herein. Thecentral processing unit 121 may utilize instruction level parallelism,thread level parallelism, different levels of cache, and multi-coreprocessors. A multi-core processor may include two or more processingunits on a single computing component. Examples of a multi-coreprocessors include the AMD PHENOM IIX2, INTEL CORE i5 and INTEL CORE i7.

Main memory unit 122 may include one or more memory chips capable ofstoring data and allowing any storage location to be directly accessedby the microprocessor 121. Main memory unit 122 may be volatile andfaster than storage 128 memory. Main memory units 122 may be Dynamicrandom access memory (DRAM) or any variants, including static randomaccess memory (SRAM), Burst SRAM or SynchBurst SRAM (BSRAM), Fast PageMode DRAM (FPM DRAM), Enhanced DRAM (EDRAM), Extended Data Output RAM(EDO RAM), Extended Data Output DRAM (EDO DRAM), Burst Extended DataOutput DRAM (BEDO DRAM), Single Data Rate Synchronous DRAM (SDR SDRAM),Double Data Rate SDRAM (DDR SDRAM), Direct Rambus DRAM (DRDRAM), orExtreme Data Rate DRAM (XDR DRAM). In some embodiments, the main memory122 or the storage 128 may be non-volatile; e.g., non-volatile readaccess memory (NVRAM), flash memory non-volatile static RAM (nvSRAM),Ferroelectric RAM (FeRAM), Magnetoresistive RAM (MRAM), Phase-changememory (PRAM), conductive-bridging RAM (CBRAM),Silicon-Oxide-Nitride-Oxide-Silicon (SONOS), Resistive RAM (RRAM),Racetrack, Nano-RAM (NRAM), or Millipede memory. The main memory 122 maybe based on any of the above described memory chips, or any otheravailable memory chips capable of operating as described herein. In theembodiment shown in FIG. 1C, the processor 121 communicates with mainmemory 122 via a system bus 150 (described in more detail below). FIG.1D depicts an embodiment of a computing device 100 in which theprocessor communicates directly with main memory 122 via a memory port103. For example, in FIG. 1D the main memory 122 may be DRDRAM.

FIG. 1D depicts an embodiment in which the main processor 121communicates directly with cache memory 140 via a secondary bus,sometimes referred to as a backside bus. In other embodiments, the mainprocessor 121 communicates with cache memory 140 using the system bus150. Cache memory 140 typically has a faster response time than mainmemory 122 and is typically provided by SRAM, BSRAM, or EDRAM. In theembodiment shown in FIG. 1D, the processor 121 communicates with variousI/O devices 130 via a local system bus 150. Various buses may be used toconnect the central processing unit 121 to any of the I/O devices 130,including a PCI bus, a PCI-X bus, or a PCI-Express bus, or a NuBus. Forembodiments in which the I/O device is a video display 124, theprocessor 121 may use an Advanced Graphics Port (AGP) to communicatewith the display 124 or the I/O controller 123 for the display 124. FIG.1D depicts an embodiment of a computer 100 in which the main processor121 communicates directly with I/O device 130 b or other processors 121′via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technology.FIG. 1D also depicts an embodiment in which local busses and directcommunication are mixed: the processor 121 communicates with I/O device130 a using a local interconnect bus while communicating with I/O device130 b directly.

A wide variety of I/O devices 130 a-130 n may be present in thecomputing device 100. Input devices may include keyboards, mice,trackpads, trackballs, touchpads, touch mice, multi-touch touchpads andtouch mice, microphones, multi-array microphones, drawing tablets,cameras, single-lens reflex camera (SLR), digital SLR (DSLR), CMOSsensors, accelerometers, infrared optical sensors, pressure sensors,magnetometer sensors, angular rate sensors, depth sensors, proximitysensors, ambient light sensors, gyroscopic sensors, or other sensors.Output devices may include video displays, graphical displays, speakers,headphones, inkjet printers, laser printers, and 3D printers.

Devices 130 a-130 n may include a combination of multiple input oroutput devices, including, e.g., Microsoft KINECT, Nintendo Wiimote forthe WIT, Nintendo WII U GAMEPAD, or Apple IPHONE. Some devices 130 a-130n allow gesture recognition inputs through combining some of the inputsand outputs. Some devices 130 a-130 n provides for facial recognitionwhich may be utilized as an input for different purposes includingauthentication and other commands. Some devices 130 a-130 n provides forvoice recognition and inputs, including, e.g., Microsoft KINECT, SIRIfor IPHONE by Apple, Google Now or Google Voice Search.

Additional devices 130 a-130 n have both input and output capabilities,including, e.g., haptic feedback devices, touchscreen displays, ormulti-touch displays. Touchscreen, multi-touch displays, touchpads,touch mice, or other touch sensing devices may use differenttechnologies to sense touch, including, e.g., capacitive, surfacecapacitive, projected capacitive touch (PCT), in-cell capacitive,resistive, infrared, waveguide, dispersive signal touch (DST), in-celloptical, surface acoustic wave (SAW), bending wave touch (BWT), orforce-based sensing technologies. Some multi-touch devices may allow twoor more contact points with the surface, allowing advanced functionalityincluding, e.g., pinch, spread, rotate, scroll, or other gestures. Sometouchscreen devices, including, e.g., Microsoft PIXELSENSE orMulti-Touch Collaboration Wall, may have larger surfaces, such as on atable-top or on a wall, and may also interact with other electronicdevices. Some I/O devices 130 a-130 n, display devices 124 a-124 n orgroup of devices may be augment reality devices. The I/O devices may becontrolled by an I/O controller 123 as shown in FIG. 1C. The I/Ocontroller may control one or more I/O devices, such as, e.g., akeyboard 126 and a pointing device 127, e.g., a mouse or optical pen.Furthermore, an I/O device may also provide storage and/or aninstallation medium 116 for the computing device 100. In still otherembodiments, the computing device 100 may provide USB connections (notshown) to receive handheld USB storage devices. In further embodiments,an I/O device 130 may be a bridge between the system bus 150 and anexternal communication bus, e.g. a USB bus, a SCSI bus, a FireWire bus,an Ethernet bus, a Gigabit Ethernet bus, a Fibre Channel bus, or aThunderbolt bus.

In some embodiments, display devices 124 a-124 n may be connected to I/Ocontroller 123. Display devices may include, e.g., liquid crystaldisplays (LCD), thin film transistor LCD (TFT-LCD), blue phase LCD,electronic papers (e-ink) displays, flexile displays, light emittingdiode displays (LED), digital light processing (DLP) displays, liquidcrystal on silicon (LCOS) displays, organic light-emitting diode (OLED)displays, active-matrix organic light-emitting diode (AMOLED) displays,liquid crystal laser displays, time-multiplexed optical shutter (TMOS)displays, or 3D displays. Examples of 3D displays may use, e.g.stereoscopy, polarization filters, active shutters, or autostereoscopy.Display devices 124 a-124 n may also be a head-mounted display (HIVID).In some embodiments, display devices 124 a-124 n or the correspondingI/O controllers 123 may be controlled through or have hardware supportfor OPENGL or DIRECTX API or other graphics libraries.

In some embodiments, the computing device 100 may include or connect tomultiple display devices 124 a-124 n, which each may be of the same ordifferent type and/or form. As such, any of the I/O devices 130 a-13Onand/or the I/O controller 123 may include any type and/or form ofsuitable hardware, software, or combination of hardware and software tosupport, enable or provide for the connection and use of multipledisplay devices 124 a-124 n by the computing device 100. For example,the computing device 100 may include any type and/or form of videoadapter, video card, driver, and/or library to interface, communicate,connect or otherwise use the display devices 124 a-124 n. In oneembodiment, a video adapter may include multiple connectors to interfaceto multiple display devices 124 a-124 n. In other embodiments, thecomputing device 100 may include multiple video adapters, with eachvideo adapter connected to one or more of the display devices 124 a-124n. In some embodiments, any portion of the operating system of thecomputing device 100 may be configured for using multiple displays 124a-124 n. In other embodiments, one or more of the display devices 124a-124 n may be provided by one or more other computing devices 100 a or100 b connected to the computing device 100, via the network 104. Insome embodiments software may be designed and constructed to use anothercomputer's display device as a second display device 124 a for thecomputing device 100. For example, in one embodiment, an Apple iPad mayconnect to a computing device 100 and use the display of the device 100as an additional display screen that may be used as an extended desktop.One ordinarily skilled in the art will recognize and appreciate thevarious ways and embodiments that a computing device 100 may beconfigured to have multiple display devices 124 a-124 n.

Referring again to FIG. 1C, the computing device 100 may comprise astorage device 128 (e.g. one or more hard disk drives or redundantarrays of independent disks) for storing an operating system or otherrelated software, and for storing application software programs such asany program related to the software for the education instrumentation(EI) system 120. Examples of storage device 128 include, e.g., hard diskdrive (HDD); optical drive including CD drive, DVD drive, or BLU-RAYdrive; solid-state drive (SSD); USB flash drive; or any other devicesuitable for storing data. Some storage devices may include multiplevolatile and non-volatile memories, including, e.g., solid state hybriddrives that combine hard disks with solid state cache. Some storagedevice 128 may be non-volatile, mutable, or read-only. Some storagedevice 128 may be internal and connect to the computing device 100 via abus 150. Some storage device 128 may be external and connect to thecomputing device 100 via a I/O device 130 that provides an external bus.Some storage device 128 may connect to the computing device 100 via thenetwork interface 118 over a network 104, including, e.g., the RemoteDisk for MACBOOK AIR by Apple. Some client devices 100 may not require anon-volatile storage device 128 and may be thin clients or zero clients102. Some storage device 128 may also be used as an installation device116, and may be suitable for installing software and programs.Additionally, the operating system and the software can be run from abootable medium, for example, a bootable CD, e.g. KNOPPIX, a bootable CDfor GNU/Linux that is available as a GNU/Linux distribution fromknoppix.net.

Client device 100 may also install software or application from anapplication distribution platform. Examples of application distributionplatforms include the App Store for iOS provided by Apple, Inc., the MacApp Store provided by Apple, Inc., GOOGLE PLAY for Android OS providedby Google Inc., Chrome Webstore for CHROME OS provided by Google Inc.,and Amazon Appstore for Android OS and KINDLE FIRE provided byAmazon.com, Inc. An application distribution platform may facilitateinstallation of software on a client device 102. An applicationdistribution platform may include a repository of applications on aserver 106 or a cloud 108, which the clients 102 a-102 n may access overa network 104. An application distribution platform may includeapplication developed and provided by various developers. A user of aclient device 102 may select, purchase and/or download an applicationvia the application distribution platform.

Furthermore, the computing device 100 may include a network interface118 to interface to the network 104 through a variety of connectionsincluding, but not limited to, standard telephone lines LAN or WAN links(e.g., 802.11, T1, T3, Gigabit Ethernet, Infiniband), broadbandconnections (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet,Ethernet-over-SONET, ADSL, VDSL, BPON, GPON, fiber optical includingFiOS), wireless connections, or some combination of any or all of theabove. Connections can be established using a variety of communicationprotocols (e.g., TCP/IP, Ethernet, ARCNET, SONET, SDH, Fiber DistributedData Interface (FDDI), IEEE 802.11a/b/g/n/ac CDMA, GSM, WiMax and directasynchronous connections). In one embodiment, the computing device 100communicates with other computing devices 100′ via any type and/or formof gateway or tunneling protocol e.g. Secure Socket Layer (SSL) orTransport Layer Security (TLS), or the Citrix Gateway Protocolmanufactured by Citrix Systems, Inc. of Ft. Lauderdale, Fla. The networkinterface 118 may comprise a built-in network adapter, network interfacecard, PCMCIA network card, EXPRESSCARD network card, card bus networkadapter, wireless network adapter, USB network adapter, modem or anyother device suitable for interfacing the computing device 100 to anytype of network capable of communication and performing the operationsdescribed herein.

A computing device 100 of the sort depicted in FIGS. 1B and 1C mayoperate under the control of an operating system, which controlsscheduling of tasks and access to system resources. The computing device100 can be running any operating system such as any of the versions ofthe MICROSOFT WINDOWS operating systems, the different releases of theUnix and Linux operating systems, any version of the MAC OS forMacintosh computers, any embedded operating system, any real-timeoperating system, any open source operating system, any proprietaryoperating system, any operating systems for mobile computing devices, orany other operating system capable of running on the computing deviceand performing the operations described herein. Typical operatingsystems include, but are not limited to: WINDOWS 2000, WINDOWS Server2012, WINDOWS CE, WINDOWS Phone, WINDOWS XP, WINDOWS VISTA, and WINDOWS7, WINDOWS RT, and WINDOWS 8 all of which are manufactured by MicrosoftCorporation of Redmond, Wash.; MAC OS and iOS, manufactured by Apple,Inc. of Cupertino, Calif.; and Linux, a freely-available operatingsystem, e.g. Linux Mint distribution (“distro”) or Ubuntu, distributedby Canonical Ltd. of London, United Kingdom; or Unix or other Unix-likederivative operating systems; and Android, designed by Google, ofMountain View, Calif., among others. Some operating systems, including,e.g., the CHROME OS by Google, may be used on zero clients or thinclients, including, e.g., CHROMEBOOKS.

The computer system 100 can be any workstation, telephone, desktopcomputer, laptop or notebook computer, netbook, ULTRABOOK, tablet,server, handheld computer, mobile telephone, smartphone or otherportable telecommunications device, media playing device, a gamingsystem, mobile computing device, or any other type and/or form ofcomputing, telecommunications or media device that is capable ofcommunication. The computer system 100 has sufficient processor powerand memory capacity to perform the operations described herein. In someembodiments, the computing device 100 may have different processors,operating systems, and input devices consistent with the device. TheSamsung GALAXY smartphones, e.g., operate under the control of Androidoperating system developed by Google, Inc. GALAXY smartphones receiveinput via a touch interface.

In some embodiments, the computing device 100 is a gaming system. Forexample, the computer system 100 may comprise a PLAYSTATION 3, orPERSONAL PLAYSTATION PORTABLE (PSP), or a PLAYSTATION VITA devicemanufactured by the Sony Corporation of Tokyo, Japan, a NINTENDO DS,NINTENDO 3DS, NINTENDO WII, or a NINTENDO WII U device manufactured byNintendo Co., Ltd., of Kyoto, Japan, an XBOX 360 device manufactured bythe Microsoft Corporation of Redmond, Wash.

In some embodiments, the computing device 100 is a digital audio playersuch as the Apple IPOD, IPOD Touch, and IPOD NANO lines of devices,manufactured by Apple Computer of Cupertino, Calif. Some digital audioplayers may have other functionality, including, e.g., a gaming systemor any functionality made available by an application from a digitalapplication distribution platform. For example, the IPOD Touch mayaccess the Apple App Store. In some embodiments, the computing device100 is a portable media player or digital audio player supporting fileformats including, but not limited to, MP3, WAV, M4A/AAC, WMA ProtectedAAC, AIFF, Audible audiobook, Apple Lossless audio file formats and.mov, .m4v, and .mp4 MPEG-4 (H.264/MPEG-4 AVC) video file formats.

In some embodiments, the computing device 100 is a tablet e.g. the IPADline of devices by Apple; GALAXY TAB family of devices by Samsung; orKINDLE FIRE, by Amazon.com, Inc. of Seattle, Wash. In other embodiments,the computing device 100 is a eBook reader, e.g. the KINDLE family ofdevices by Amazon.com, or NOOK family of devices by Barnes & Noble, Inc.of New York City, N.Y.

In some embodiments, the communications device 102 includes acombination of devices, e.g. a smartphone combined with a digital audioplayer or portable media player. For example, one of these embodimentsis a smartphone, e.g. the IPHONE family of smartphones manufactured byApple, Inc.; a Samsung GALAXY family of smartphones manufactured bySamsung, Inc.; or a Motorola DROID family of smartphones. In yet anotherembodiment, the communications device 102 is a laptop or desktopcomputer equipped with a web browser and a microphone and speakersystem, e.g. a telephony headset. In these embodiments, thecommunications devices 102 are web-enabled and can receive and initiatephone calls. In some embodiments, a laptop or desktop computer is alsoequipped with a webcam or other video capture device that enables videochat and video call.

In some embodiments, the status of one or more machines 102, 106 in thenetwork 104 is monitored, generally as part of network management. Inone of these embodiments, the status of a machine may include anidentification of load information (e.g., the number of processes on themachine, central processing unit (CPU) and memory utilization), of portinformation (e.g., the number of available communication ports and theport addresses), or of session status (e.g., the duration and type ofprocesses, and whether a process is active or idle). In another of theseembodiments, this information may be identified by a plurality ofmetrics, and the plurality of metrics can be applied at least in parttowards decisions in load distribution, network traffic management, andnetwork failure recovery as well as any aspects of operations of thepresent solution described herein. Aspects of the operating environmentsand components described above will become apparent in the context ofthe systems and methods disclosed herein.

B. Computer Environment for Instrumentation of Education Processes

FIGS. 2A and 2B show charts illustrating, respectively, the averagespending per student and the numeracy proficiency scores for a pluralityof countries. By comparing the ranking of any given country in terms ofthe average spending per student to the respective ranking of the samecountry in terms of the numeracy proficiency score, one can see thatincreased spending per student does not necessarily translate intohigher academic performance. For example, the United States of Americawhile having the second highest spending amount per student (more than$90,000 per student) is only ranked 21^(st) in terms of the numeracyproficiency scores. In contrast to the United States of America, theSlovak Republic spends only about $23,000 per student and is ranked8^(th) in terms of the numeracy proficiency scores.

Considering the spending amount per student data and the numeracyproficiency scores shown in FIGS. 2A and 2B, one can conclude thattransforming education or education systems to improve academicperformance and increase student learning outcomes may not be achievedsimply by increasing money spending on education. In fact, an educationsystem is a very complex environment with a multitude of factors thatinfluence the education process (e.g., education policies, educationstrategies, and/or education operations) and student performance. Suchfactors interact with each other, and are inter-dependent, in a complexand dynamic way. The complexity of the education process makes itsubstantially difficult to construct an information model that describesfully and accurately a real word education system (or educationprocess), can accommodate various circumstances, and can providediagnostic or interventional analysis.

An educational system as a multilevel environment can include factorsrelated to students, one or more schools, one or more school districts,one or more states, or one or more countries associated with thateducational system. However, many existing studies of educationalsystems do not typically take into account the full spectrum ofpotential factors and levels due to the complexity of the challenge. Onthe other hand innovation and technology can play a significant role incollecting, storing, and transforming the data achieved from educationsystems to a level where they can guide and help in daily decisionmaking processes and other purposes. This is especially possible, whenthe used models are white-box-type of models enabling persons not havingstatistical background to interpret and use the model outcomes.

Computerized education instrumentation (EPI) systems and methods of thecurrent disclosure include one or more processors generating one or moremodels of a multitude of factors (or variables) that can influenceeducation processes. The one or more processors can translate thegenerated models into intuitive visual (or graphical) representations tohelp understand the relative importance of the variables and theirinter-dependencies. Contrary to classical statistical analysis methodswhich may formulate complex relationships among variables as high-degreeequations, the graphical representations corresponding to the generatedmodels can enable people with non-technical background to easilycomprehend the degree of influence of various variables on the educationprocess and the learning outcomes. Based on the generated models, theone or more processors can provide predictions of learning outcomes, andenable educated scenario planning to improve the performance of theeducation process. A computer platform (also referred to as EDLIGO′platform) implementing the education instrumentation systems and methodsof the current disclosure can enable observability and educatedcontrollability of the education process(es).

The education process can be defined, or described, in terms of aninformation theory perspective, where students act as “receivers” andeducation agencies (e.g., government, ministry of education, schooldistrict, school board, teacher/professor, etc.) act as “transmitters.”The content of, and goals behind, a curriculum, program, or course thatis set by one or more education agencies (e.g., “transmitter(s)”) canrepresent the information sent from the “transmitter(s)” to the“receivers” (e.g., students). The difference between the curriculum,program, or course as set by the education agency and the learningoutcome (e.g., learning that students achieved and can reliablydemonstrate at a given time point of the education process) representthe information loss between the “transmitter(s)” and the “receivers.”The information loss is caused by at least a subset of the multitude ofvariables related to the education process and/or the educationsystem(s). By modeling these variables, the education instrumentationsystems and methods of the current disclosure can identify the relativecontribution of various factors (or variables) to the increase ordecrease of information loss between the “transmitter(s)” and“receivers.” Using generated models, the education instrumentationsystems and methods of the current disclosure can predict learningoutcomes and/or recommend one or more action plans to improve theperformance of education processes or education systems.

Referring to FIG. 3, an education instrumentation (EI) computer platformcan include an education instrumentation (EI) system 300, a plurality ofexternal data bases 302 a-m (referred to hereinafter individually orcollectively as external database(s) 302), and a plurality of clientdevices 102 a-n (also referred to individually or collectively as clientdevice(s) 102). The EI system 300 can include a data collector 304, adata preprocessor 306, a database 308, an analysis module 310, a libraryof validated models 312, a simulation module 314, and an output module316.

The EI system 300 can include, or can be implemented on, one or morecomputing devices, such as server(s) 106 (shown in FIG. 1A) or cloud 108(shown in FIG. 1B). The EI system 300 can be communicatively coupled theexternal databases 302 and the client devices 102 through one or morecommunication networks, such as network 104 described above with regardto FIG. 1A.

The external databases 302 can include one or more student informationsystems (SISs), one or more administration databases, one or moreeducation benchmarking data sources, one or more learning managementsystem (LMS) data sources, one or more assessment data sources, or acombination thereof. The data collector 304 can receive (or import) avariety of data sets, from the external databases 302, related tovarious aspects of education. The data collector 304 can include one ormore application program interfaces (APIs) for accessing the externaldata bases 302. The data collector 304 can receive student informationdata (e.g., name, ID, age, gender, parents' education level(s), parents'occupations, social/cultural/economic background information, academicperformance, behavior, attendance, etc.), class or grade informationdata (e.g., class size(s), teacher-student ratio, extra curriculumactivities, etc.), books' information data (e.g., books used in eachsubject), educational applications' information data (e.g., computer ormobile applications used in school), school facilities' informationdata, or a combination thereof from the student information system(s).An Administration database can provide data related to qualifications,skills, professional development or paygrades of principals, teachers,and/or staff associated with one or more schools. Administrationdatabases can also provide budgeting and expenditure information data,such as education budgets for various school districts, schools budgets,programs' budgets.

The education benchmarking data source(s) can include websites ordatabases associated with the Program for International StudentAssessment (PISA), Trends in International Mathematics and Science Study(TIMSS), Progress in International Reading Literacy Study (PIRLS),Smarter Balanced Assessment Consortium in USA, other National orInternational Benchmarking organizations/institutions (e.g., StateDepartments of Education, OECD, World Bank, or UNESCO), or a combinationthereof.

The PISA data is generated through a worldwide study by the Organizationfor Economic Co-operation and Development (OECD) in member andnon-member nations of 15-year-old school pupils' scholastic performanceon mathematics, science, and reading. It was first performed in 2000 andthen repeated every three years. It is done with a view to improvingeducation policies and outcomes. It measures problem solving andcognition in daily life. 470,000 15-year-old students representing 65nations and territories participated in PISA 2009. An additional 50,000students representing nine nations were tested in 2010.

The Trends in International Mathematics and Science Study (TIMSS) datasources provide a series of international assessments of the mathematicsand science knowledge of students around the world. The participatingstudents come from a diverse set of educational systems (countries orregional jurisdictions of countries) in terms of economic development,geographical location, and population size. In each of the participatingeducational systems, a minimum of 4,500 to 5,000 students are evaluated.Furthermore, for each student, contextual data on the learningconditions in mathematics and science are collected from theparticipating students, their teachers and their principals via separatequestionnaires. TIMSS is one of the studies established by theInternational Association for the Evaluation of Educational Achievement(IEA) aimed at allowing educational systems to compare students'educational achievement and learn from the experiences of others indesigning effective education policy. This study was first conducted in1995, and has been performed every 4 years thereafter. In most of thecycles the study assesses 4th and 8th grade students. Therefore, some ofthe participating educational systems have trend data across assessmentsfrom 1995 to 2011.

The Progress in International Reading Literacy Study (PIRLS) is aninternational study of reading achievement for fourth graders. It isconducted by the International Association for the Evaluation ofEducational Achievement (IEA). The studies are designed to measurestudents' reading literacy achievement, to provide a baseline for futurestudies of trends in achievement, and to gather information aboutstudents' home and school experiences in learning to read. PIRLS 2006tested 215,000 students from 46 educational systems. PIRLS 2011 (themost recent cycle) testing has been done and the results were publishedDecember 2012.

LMS data sources can include websites or databases providing informationrelated to curriculum standards (e.g. Common Core), learning objectives,course design information, or a combination thereof. Assessment datasources can include websites or databases that provide standardizedtests scores/results (e.g. Smarter Balanced scores).

The data collector 304 can receive data from client devices 102. Forinstance, the data collector 304 can receive lesson planning data,teacher notes data from client devices associated with teachers,professors, or educational staff. The data collector 304 can receiveother data, such as surveys (e.g. student surveys, community surveys,parents feedback), or other manually input data (e.g., spreadsheets),social media data from social networks, or other online data (e.g., fromteacher or school rating websites).

The data preprocessor 306 can preprocess data received by the datacollector 304 and store the preprocessed data in the database 308. Thedata preprocessor 306 can delete or remove incomplete data (e.g., datawith many missing values), incorrect data (e.g., remove outliers byusing well known proximity-based outlier detection algorithm),improperly formatted data (e.g., due to corruption), data redundancies(e.g., remove data duplicates). For example, many of the individualsurvey question (or respective indices) can contain often a largeportion of missing values, which makes them useless in the analysis.Therefore, while certain indices, with a significant portion of missingvalues, may be interesting, they may not be reliable with regard to dataanalysis due to a large number of missing values (e.g., higher than30%). The data preprocessor 306 can estimate missing values in one ormore data sets (e.g. using Expected Maximization algorithm). The datapreprocessor 306 can normalize data (e.g., by applying weighting) toremove or alleviate any data biases or can smooth some data values(e.g., by averaging) in order to find trends. The data preprocessor 306can also merge various data elements received from different sourcestogether. The data preprocessor 306 can store preprocessed data in thedatabase 308.

The database 308 can be configured to store preprocessed data accordingto one or more specific formats. For instance, the database 308 canorganize various data sets according to various variables (or factors)having potential influence on student performance. Such variables (orfactors) can include student behavior, student's social, cultural oreconomic background, parents' educational level, family structure,school size, school resources, class size, student-teacher ratio,teacher qualifications, extra-curriculum activities, or a combinationthereof. The database 308 can classify preprocessed data sets based on,for example, respective data sources, dates, or geographical areas.

The analysis module 310 can analyze the collected data (e.g., thepreprocessed data stored in database 308) to generate a plurality ofmodels that represent one or more education processes. The generatedmodels can include mathematical or statistical models that simulate theimpact or effect of various factors or variables in the collected dataon various learning outcomes. For example, the analysis module 310 canbe configured to generate Bayesian networks, directed acyclic graphs,neural networks, or convolutional networks. Each generated model canillustrate the inter-dependencies between distinct data variables andthe contribution of each data variable to a learning outcome for a giveneducation process. As an illustrative example, a generated model canillustrate the inter-dependencies between factors such as studentbehavior, student's social, cultural or economic background, parents'educational level, family structure, school size, school resources,class size, student-teacher ratio, teacher qualifications,extra-curriculum activities, and how these factors affect learning outcome in math.

The analysis module 310, or the generated models, may identify importantfactors (or dominant variables) with respect to affecting the academicperformance of students in one or more subjects. Identifying suchdominant data variables (or important factors) can allow teachers,faculty, principals, superintendents and policy makers to make educatedconclusions and take the “proper” action(s) to maximize positive impacton student learning outcomes. The analysis module 310 can be configuredto perform deep and objective data analysis (e.g., with non-bias) ofcollected education data and provide visually meaningful data models(e.g., Bayesian networks, directed acyclic networks, neural networks,convolutional networks) analytics representative of key factors (such asdominant factors) that can predict and often also directly influence theacademic performance at the individual or group level. The analysismodule 310 can use, for example, standards frameworks data (e.g., PISAdata, TIMSS data, PERLS data, or Smarter Balanced Assessment Consortiumdata), curriculum or lesson planning data, assessment data, professionaldevelopment data, student engagement data, personalized learningexperience data, SIS data, LMS data, budgeting data, etc., as trainingdata to generate the models. The generated models can determinedependencies (or correlations) between variables across various datatypes or data sets, and quantify (or estimate) the impact of suchvariables on learning outcomes.

FIG. 4 shows a block diagram illustrating an architecture of theanalysis module 310, according to one or more embodiments. The analysismodule 310 can include a data selector 402, a model generator 404, alibrary of modeling algorithms 406, a model validator 408, and a modelperformance metrics' file or data structure.

The data selector 402 can select one or more data subsets from the datasets stored in database 308 for use as training data to generate a givenmodel or as test data to validate a generated model. For example, thedata selector can select one or more subsets of PISA data, TIMSS data,PIRLS data, Smarter Balanced Assessment Consortium data, or otherNational or International Benchmarking data stored in the database 308for use as training data. For example, the PISA data sets stored in thedatabase 308 can include results from surveys organized by OECD andconducted by education authorities in different countries and fordifferent years (e.g., 2006, 2009, 2012, 2015). The data selector 402can select a subset of these data sets, for example, related to aspecific year (e.g., 20115), a specific country (e.g., U.S.), and/or aspecific subject or field (e.g., math, science or reading). The dataselector 402 can also select data subset(s) from data sets includingbackground information obtained from students and schools, individualsurvey results from students and schools, performance result inmathematics, science and reading for a plurality of students, indicesrepresenting summaries that describe sets of individual surveys (theseindices are defined and pre-calculated by OECD authorities), statisticaldata associated with students, schools or school districts (e.g., forweighting the selected data to avoid bias in analysis).

In selecting training or test data, the data selector can take intoaccount the total number of variables or the type of variables to use ingenerating a model. For example, data sets (such as PISA, TIMMS, PERLSor Smarter Balanced Assessment Consortium data sets) can include about500 or even more variables per data set with significantmulti-collinearity between many variables (correlations or dependenciesbetween variables in the same set). These correlations or dependenciesbetween such large number of variables can set serious challenges (e.g.,convergence and/or computational challenges) in any statisticalanalysis. Also, the type of variables may vary, ranging from binary tonumerical values. Such variation may create some restrictions withregard to the usage of certain algorithms.

In some embodiments of the current disclosure, the data selector 402 canbe configured to reduce (or limit) the number of variables consideredfor generating models to reduce the complexity of the system and thecomplexity of the generated models. For example, the data selector 402can select data variables representing indices (e.g., each defined basedon multiple individual questions and describes a certain theme on awider scope) rather than variables representing individual surveyquestions. The use of indices can decrease the number of variables inthe model from about 500 to about 70 variables. In some implementations,the data selector 402 may select variables (e.g., related to individualsurvey questions) instead of the indices for model generation or modelvalidation while employing the same processing methodology as in thecase where the indices are used.

The model generator 404 can use the data subsets selected by the dataselector 402 as training data to generate a model. The model generator404 can select an algorithm from a plurality of algorithms stored in thelibrary of modeling algorithms 406. The model generator 404 can executethe selected algorithm using data subsets selected by the data selector402. The plurality of algorithms can include, for example, score-basedlearning algorithms (e.g., MDL scoring), the TABOO algorithm, the TreeAugmented Naïve Bayes (TAN), or other algorithms for identifying thestructure of a model representing a set of data.

For example, the model generator 404 can use a version of the TABOOalgorithm to generate a descriptive model or use the Tree AugmentedNaïve Bayes (TAN), algorithm to generate a predictive model. Descriptiveand predictive models can illustrate the dominant data variables (ordominant factors) and the dependencies between the data variables. Adominant data variable (also referred to as important factor/variable)can be a data variable that is a relatively strong predictor (e.g.,compared to other data variables) for a target (e.g., academicperformance or learning outcome for one or more students in one or moresubjects). In other words, dominant data variables carry or reveal mostof the information about a target.

The model generator 404 can compute the strength in predicting a target(e.g., learning outcome) as a Pearson correlation between the predictor(e.g., a data variable), or as a Kullback-Leibler divergence. TheKullback-Leibler divergence can support nonlinear relationships. In mostcases, the generated model can include a few strong predictors (ordominant data variables) and a long tail (or large number) of lessstrong predictors. The difference between dominant and non-dominant datavariables is based on the dependency level as a Fullback-Leiblerdivergence value (e.g., compared to a threshold value of 6%). Forexample, when the value is more than 6%, the predictor (or datavariable) can be considered as dominant. For example, FIG. 5 shows achart illustrating the predicting strength (y-axis) of a plurality ofvariables (x-axis) with regard to student learning outcome in sciencefor Germany in 2009. The model generator can use, for example, thelevels 6% and above to identify the dominant variables, and the levels2.5%-6% to identify useful variables.

The model generator 404 may employ a minimum description length (MDL)scoring based algorithm to generate descriptive models. Descriptivemodels generated by the analysis module 310 can be represented throughvisually comprehensible graphical objects (e.g., graphs) that helpclient device users to visualize and comprehend the most relevantattributes (variables) and their dependencies. For example, thegraphical object can include a network graph including a plurality ofnodes representing data variables associated with a predicting strengthgreater than or equal to a given threshold value (e.g., 2.5%) and edgesrepresenting dependencies between pairs of data variables. A descriptivemodel can be viewed as a generic dependency map that describes therelations between data variables and indicates visually to stakeholders(or users) the important (or dominant) variables (e.g., in terms ofinfluencing academic performances) in the variable space. Thedescriptive models can provide the stakeholders with a quick view aboutimportant variables and their relationships and indicates potentialpaths to reach a variable B from a variable A.

The model generator 404 may employ a Tree Augmented Naive Bayes (TAN)algorithm to generate predictive models. The TAN algorithm relaxes theNaive Bayes attribute independence assumption by employing a treestructure, where each attribute only depends on the class and one otherattribute. The model generator 404 can construct a predictive model, forexample, based on the TAN algorithm for use to predict one or moretargets. A target (e.g., performance or learning outcome either inmathematics, science or reading) can be predicted and explained based ondependencies with the other variables in the dataset.

The model validator 408 can validate the model generated by the modelgenerator 404 using one or more test data subsets and one or more modelperformance metrics stored in one or more data files (or datastructures) 410. The data selector 402 can select the test datasubset(s) from data stored in the database 308 (similar to the selectionof the training data subset(s) for generating the same model). Theselected test data subset(s) can be different than the selected trainingdata subsets used to generate the model. The model validator 408 can beconfigured to check the accuracy of the generated model using the testdata and the performance metrics. The performance metrics can include,for example, error tolerance values. The model validator 408 can, forexample, compare learning outcome output by the generated model tolearning outcome values in the test data to check the accuracy of thegenerated model. Upon validating the generated model, the modelvalidator 408 can store the validated model in library of validatedmodels 312. The library of validated models 312 can store a plurality ofmodels generated and validated by the analysis module 310.

Referring back to FIG. 3, the analysis module 310 can generate aplurality of models in a similar way described with regard to FIG. 4 andstore these models in library of validated models 312. The plurality ofmodels can be generated based on different subsets of training data andvalidated using different subsets of test data. The plurality of modelscan be associated with different student populations (e.g., associatedwith a class, a grade level in a school, a school, school district, astate, or a country), different subjects (e.g., math, science, English,etc.), or other aspects of education systems. The analysis module can beconfigured to generate one or more models responsive to one or morerequests from client devices 102 (e.g., associated with asuperintendent, a principal, or a teacher).

Responsive to a request from a client device 102 (e.g., a request topredict or compute a learning outcome or a performance metric associatedwith the education process), the simulator 314 can select a model fromthe plurality of models stored in the library of validated models 312.The simulator 314 can select the model based on an indication of aselected model, a selected subject, a selected student population, or acombination thereof, in the request received from the client device 102.The simulator 314 can also select a data subset (e.g., SIS data, datareceived from one or more teachers, data from a one or more schools,data from a school district, etc.) from the database 308. The simulator314 can used the selected data subset and the selected model to computethe performance metric (e.g., a learning outcome). The simulator 314 canbe configured to simulate a specific model on a regular basis based on,for example, data (e.g., lesson planning data, teacher feedback orteacher notes dates, or other data) received regularly from clientdevices 102. As such, the EI system 300 can allow monitoring theperformance of the education process as it progresses. For example, theEI system (or EDLIGO) can provide estimates or predictions of students'performance(s) in mathematics, science or reading throughout a lifecycle of the education process.

The output model 316 can be configured to provide the computedperformance metric for display on the client device(s) 102. The outputmodel 316 can also provide other data, such as data collected and storedin the database 308, for display on client devices 102. The outputmodule 315 can provide graphical representations of generated models orother data for display on client devices 102.

FIG. 6 shows a Bayesian network model generated by the EI system 300,according to one or more embodiments. The Bayesian network model caninclude a plurality of nodes (shown as circles in FIG. 6) with pairs ofnodes connected via edges. The nodes represent the data variables. Thesize of each node represents the dominance of the respective variable,whereas the color of the node represents a category of the respectivevariable. The bigger the circle, the more dominant is the correspondingdata variable. The edges between pairs of nodes represent thedependencies between corresponding pairs of data variables. For example,the edges can be indicative of a causality relationship betweendifferent data variables (or factors). The analysis module 310 canestimate causality between any pair of data variables based on theconditional probability between that pair of data variables.

Bayesian networks can unveil dependencies between data variables (e.g.,representing learning attributes or other attributes) which act aspredictors of student performance, for example, in mathematics, scienceor reading. A real word education system can be a very complexmultilayer environment that is difficult to model by accounting for allthe factors affecting to learning outcome. The models generated by theEI system 300, such as the Bayesian network shown in FIG. 6, can providea summary view for stakeholders (e.g., superintendents, principals,teachers, or other people involved in education) indicative of thedominant or important factors and the most relevant and importantdependencies between various factors or attributes. The EI system 300can also generate other types of models, such as directed acyclic graphmodels, neural network models or convolutional network models.

FIG. 7 shows a diagram illustrating various stages in a life cycle of aneducation process. Such stages can start, for example, with “strategicplanning (e.g., at a school district level, state level, or countrylevel), and can include stages such as “professional development,”“planning standard achievement,” “curriculum (or lesson) planning,”“assessment,” and “reporting.” The education instrumentation platform(e.g., as shown in FIG. 3) can provide one or more client applicationsrunning on client devices 102 to interact with the EI system 300. Theone or more client applications can provide a plurality of userinterfaces (UIs) associated with the plurality of stages of the lifecycle shown in FIG. 7. As such, users of the client devices 102 canmonitor the progress of the education process throughout the life cycle.

The client application(s) and the UIs provided on each client device 102can vary based on, for example, the respective user profile. Forexample, the client application or the UIs available to a teacher may bedifferent than those available to a super intendent or a principal.

FIGS. 8A-E and 8B show various UIs associated with the curriculum (orlesson) planning stage. The UI in FIG. 8A allows a user to select one ormore education standards among a plurality of standards (e.g., CommonCore State Standards). The UI in FIG. 8A allows a user to select one ormore education standards among a plurality of standards (e.g., CommonCore State Standards). Each standard can include one or more collectionsof standard elements indicative of, for example, mastery levelsexpressed in terms of a taxonomy (e.g., Bloom's taxonomy or Webb'staxonomy). FIG. 8B shows a UI for selecting a mastery level (e.g.,remember, understand, apply, analyze, evaluate, or create) for aselected standard. The mastery level can be indicative of what thestudent is expected to master with regard to a specific subject. FIG. 8Cshows a UI for selecting course or subject to which the selectedstandard and/or the selected mastery level is to be applied. FIG. 8Dshows a UI for selecting a type of assessment to be applied with regardto a selected course or subject, and FIG. 8D shows a UI for selecting atime frame by which the selected mastery level(s) is to be achieved.

FIG. 9 shows a UI associated with the instruction stage of the lifecycle in FIG. 7. In particular, the UI in FIG. 9 shows data and chartsindicative of progress in addressing selected standards throughinstructions (e.g., teaching) and assessment.

FIGS. 10A-C show various UIs associated with the reporting stage of thelife cycle in FIG. 7. For example, the UI in FIG. 10A shows performanceof a school district. The UI in FIG. 10B shows standard coverage. The UIin FIG. 10B can also be part of the instruction stage to allow trackingof standards' coverage in the middle of the life cycle. The UI in FIG.10C shows students' performances in various subjects.

Besides predicting or estimating learning outcomes, the EI system 30 cangenerate and provide recommendations (e.g., for teachers, principals, orsuperintendents) to plan or adjust an education process. For example,the simulator 314 can use a one or more validated models and one or moredata subsets selected from the database 308 to simulate variousscenarios (e.g., based on different standards, different mastery levels,or variations of other attributes). The simulator 314 can comparelearning outcomes associated with the various scenarios and provide arecommendation indicative of the best performing scenario. In someimplementations, the EI system 30 can provide the comparison of thevarious scenarios for display on one or more client devices 102.

The education instrumentation platform described herein allows leaders,e.g. district superintendents (or employees) to track curriculumcoverage at the district level (or at finer levels such as school level,grade level or class/teacher level) through a curriculum coverage UI(e.g., as shown in FIG. 10B). The education instrumentation platform canalso provide leaders/employees with real-time visibility and insights ofthe education processes (such as lesson planning, instructionassessment, educational and school resources, student/communityengagement, teacher and/or staff professional development, etc.) in therespective area in responsibility (e.g. a district). The educationinstrumentation platform can also allow leaders/employees to monitorand/or enforce alignment of educational programs in the district tostate and/or country objectives. The education instrumentation platformcan also allow leaders/employees to set focused actions, for instance,to be implemented or executed by principals, teachers or school staffand administrators.

For state leaders, the education instrumentation platform can provideUIs for presenting information indicative of global competitivenessanalysis of the education system and real-time insights of the educationprocesses. The UIs can allow state leaders to address the root causesimpacting performance and systematic governance by, for instance,inputting, modifying or deleting instructions or guidelines to beimplemented by school districts, schools, principals, teachers, schoolstaff, or a combination thereof.

The education instrumentation platform can also allow for personalizingteaching or learning techniques for various students or groups ofstudents. The personalization of teaching or learning techniques can beimplemented through the analysis module 310 and the simulator 314, whichbased on observations simulations of various scenarios can recommend tostudents, teachers, faculty, administrators and policy makers variousinterventions including (1) predicting student future performance ateach subject skill and sub-skill based on previous and currentperformance(s), recommending content suitable for students based onanalysis of previously covered content and corresponding standards, (3)guiding students in selections of subjects and electives to achieveoptimum future results, (4) designing what to assess to accuratelyevaluate students' performance and the type of assessment required, (5)recommending how learning can be individualized for more effectivelearning experiences, (6) analyzing how students proceed through acourse (pace of learning, motivation, level of interest and engagement),(7) predicting students future orientation and hence motivation, (8)recommending new and alternative learning opportunities and differentcourses to students, (9) analyzing undesirable learning behaviors, anddetect affective states (e.g., boredom, frustration) of the learner,(10) predicting attrition likelihood related to courses or programs,(11) predicting student grades and the factors that influence theperformance of the students, (12) recommending what material and humanresources will be needed over a course of time, or (13) predicting howobjectives are to be met.

The EI system 300 can provide real-time analysis results by feedingrecent (or new) data to one or more generated and validated models. TheEI system 300 can provide interactive analysis results by enabling thevalidated models to be used by students, teachers and policy makers fortheir what-if-analysis. The EI system 300 can provide reports, scenariomodelling results, and/or prediction results.

Outcomes in education are often equated to assessment results. Oftenorganization define action plans on the basis of assessment results,usually at the end of the life cycle, of the education process. Whereassuch assessment is important, it comes late in the life cycle. Thechallenge becomes is then how to understand well ahead of the assessmentresults, how students are expected/predicted to perform. The educationinstrumentation platform described herein is can employ an informationtheory paradigm of information loss between sender and receiver. Inevery stage of the life cycle depicted on the diagram in FIG. 7, thereis certain likelihood that information can get lost. For example, if astudent is absent 50% of the time, we can approximate information thatwas received at student end (receiver) is only 50% of what the teacherintended to broadcast (in the form of lesson). When considering allitems ahead of assessment in the cycle, one can approximate theinformation received at student Ireceived end as a function of all theoperations/steps before. Ireceived is an appropriate approximation ofthe expected outcomes the student is going to score during assessment.To the best of our knowledge, such comprehensive definition andapproximation together with the visualization of its results has neverbeen addressed in previous works.

As an example, consider a first scenario where a teacher plans lessonsfor 100% of the curriculum, the teacher delivers lessons for 100% of thecurriculum, a student attends 100% of the lessons, and the student iswell behaving 100%. Then we can assume that the student received 100% ofthe planned curriculum. In a second scenario, a teacher plans lessonsfor 80% of the curriculum (e.g. missed to cover 10 common core standardsout of 50), the teacher delivers lessons for 80% of the curriculum (wasabsent 20% of the time and there was no replacement), a student attends90% of the lessons, and the student is well behaving 50% (not attentive,playing in classroom). Then we can assume that the student received themultiplication of all above; e.g., 28.8% of the planned curriculum.

If Itarget: denotes the information, skills, competences, and/orlearning outcomes which the governing body (e.g. State Department ofEducation) expects students to master at a certain mastery level, by acertain point in time (e.g. by end of Grade 10) or within one semesterof study of a particular subject, Itarget: can be expressed, forexample, using three variables Itarget1, Itarget2, and Itarget3.Itarget1 can represent the standards and, in general, the information,skills, competences, and/or learning outcomes which the governing body(e.g. State Department of Education) expects students to master.Itarget2 can represent the level(s) of mastery of the items defined inItarget1. The level(s) of mastery can be expressed as a percentagenumber (e.g. 75%), as a level in a defined taxonomy (e.g., we may expectthe students to master a Common Core standard at Webb's Degree ofKnowledge Level 3, Strategic Thinking/Reasoning, or at Revised Bloom'sTaxonomy Level “Apply”), or as a combination thereof (e.g., as a 2-tupleof Webb's and Bloom's taxonomies (Strategic Thinking/Reasoning, Apply).Itarget3 can define the timeframe by which the mastery is expected, inabsolute or relative form. Itarget can be referred to as the curriculum.

Let X1 denote the mastery level of Itarget1 (e.g., of information,skills, competences, and/or learning outcomes) which the teacherspossess, as a function of their academic qualifications, professionalqualifications, professional development trainings they attendedsuccessfully, skills acquired through self-learning, or through feedback(e.g., from inspectors, leaders, peers, students, community, etc.). LetX2 be the level of coverage of the standards (Itarget1) at the expectedlevel (Itarget2) in the lesson planning through appropriate units,lessons, learning objectives and tasks. In other words, how far do thelesson plans cover the required curriculum defined by Itarget1 andItarget2 to be achieved by Itarget3. Let X3 be the degree to which whatwas planned in X2 (lesson planning) is covered/addressed throughinstruction and other teaching methods (e.g. project work). This can beexpressed as percentage of X2 or other more general function of X2(frequency of coverage of a given item of Itarget1). Let X4 be theattendance rate of a student (e.g. 90% attendance rate). Let X5 be thediscipline (expressed in percentage) of a student (e.g. only half of thetime attentive and engaged, then 50%). Let X6 be student feedback(expressed as a percentage of e.g. favorable responses), and X7 be theadditional remedial or enrichment activities performed by the student(e.g., extra curriculum activities/courses).

Ireceived can be defined as the approximated information which wasreally received at the student end and is expressed by three variablesIreceived1, Ireceived2, and Ireceived3. Based on the data pointsItarget, X1 to X7, the education instrumentation platform can apply oneor a combination of one or more processes to approximate Ireceived. Theone or more processes can include process 1, which can be adeterministic algorithm, and process 2, which can include a machinelearning process, which uses previously (e.g. from past terms or years)recorded values of Itarget, X1 to X7 and Ireceived to learn how topredict Ireceived from Itarget and X1 to X7. The EI system 300 can usethe trained model to predict Ireceived then based on new observations ofItarget and X1 to X7

According to Process 1, (Ireceived) can be calculated from the values ofItarget and X1 to X7 and is expressed as a function such asIreceived=Itarget×(X1×X2×X3×X4×X5×X6)+X7. A more general functionapplying weighting factors to each of X1 to X9 to emphasize the one orother factor can be used such as Ireceived=(wtarget·Itarget)×(w1·X1×w2·X2× w3·X3× w4·X4× w5·X5× w6·X6)+w7·X7. Other general mathematicalfunction taking as parameter Itarget and X1 to X7, i.e. a function f(Itarget, X1, . . . , X7) can also be employed.

Process 2 can include three steps; discovery phase, a learning phase,and a prediction step. In the discovery phase, the EI system 300 can usescore based algorithms to understand the dependencies between thedifferent data points (expressed for example as dependency map). In thelearning phase, the EI system 300 can train a machine learning model topredict Ireceived from Itarget and X1 to X7. In the prediction phase,the EI system 300 can predict Ireceived from a new set of recordedItarget and X1 to X7.

FIG. 11 shows a flowchart of a method 1100 for computing performancemetrics of education processes. The method 1100 can include storing, byone or more processors, training data and test data in a database (step1102). The training data and the test data can be related to studentacademic performance of a first plurality of students. The one or moreprocessors can receive student information data for a second pluralityof students from a student information system (SIS), and receive, fromone or more client devices, one or more sets of standard elements forone or more lesson plans selected via a first user interface associatedwith the one or more client devices (step 1104). The first userinterface including lesson planning templates for selection of standardelements for lesson planning. The one or more processors can generate,using the training data, a plurality of models to simulate performanceof an education process, and validate the plurality of models using thetest data (step 1106). The one or more processors can select a modelfrom the plurality of models based on a selection by a client device ofa student population among the second plurality of students (step 1108).The one or more processors can compute a performance metric using theselected model and at least a subset of the student information data andthe one or more sets of standard elements (step 1110). The one or moreprocessors can provide the computed performance metric for display onthe client device via a second user interface of the plurality of userinterfaces (1112).

The steps of the method 1100 can be carried out by the EI system 300 asdescribed above with regard to FIGS. 3-10C.

With regard to Itarget2 (e.g., the level of mastery of the standards andin general skills, competences, and/or learning outcomes which thegoverning body expects students to master), in order to preciselyunderstand the attained degree of mastery for each standard, thestandard is deconstructed and various levels of mastery are defined atsub-item level (e.g., building a hierarchy: start at atomic level andthen aggregate), as described, for example, with regard to FIGS. 8A and8B. For each of the sub-items, the EI system 300 can apply process 1and/or process 2.

When an assessment (test) gets designed, it is possible to understandand measure whether the assessment elements (e.g. quiz, exercises)called Y1, Y2, . . . Yn are effectively assessing what the studentreceived Ireceived and at which mastery level. In other words, it ispossible to build a gap analysis between whether Y1, Y2, . . . Yn arealigned to Itarget and what the students actually received and expressedas Ireceived. Further, the assessment results of Y1, Y2, . . . Yn tofurther tune the model and processes 1 and 2 outlined above. Given thata student/learner targets reflect also progress over time andassessments are conducted over a period of time, multiple times, and invarious forms (e.g. diagnostic, formative, summative), a time series ofthe targets, measurements, and outcomes are analyzed using time seriesanalysis techniques (longitudinal analysis, repetitive in time) can begenerated by the system.

While the invention has been particularly shown and described withreference to specific embodiments, it should be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the inventiondescribed in this disclosure.

While this specification contains many specific embodiment details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of particular inventions.Certain features described in this specification in the context ofseparate embodiments can also be implemented in combination in a singleembodiment. Conversely, various features described in the context of asingle embodiment can also be implemented in multiple embodimentsseparately or in any suitable subcombination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination can in some cases be excised from the combination, and theclaimed combination may be directed to a subcombination or variation ofa subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated in a single software product or packaged intomultiple software products.

References to “or” may be construed as inclusive so that any termsdescribed using “or” may indicate any of a single, more than one, andall of the described terms.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain embodiments, multitasking and parallel processingmay be advantageous.

1-20. (canceled)
 21. A system comprising: one or more processors; and amemory storing computer code instructions, which when executed by theone or more processors, cause the one or more processors to: obtaintraining data and test data related to student performance for a firstplurality of students, the training data and the test data associatedwith a plurality of variables having influence on performance of aneducation process; generate, using the training data, a machine learningmodel to simulate the performance of the education process with respectto information gain for students at one or more stages of a plurality ofstages of the education process, the machine learning model (i)representing a network of nodes corresponding to a respective subset ofvariables of the plurality of variables for simulating the performanceof the education process, and (ii) illustrating for each variable of theplurality of variables a respective strength of predicting theperformance of the education process; validate the machine learningmodel using the test data; receive education related data for a student;and simulate the machine learning model using the education related datafor the student to predict an amount of information gain for the studentat a stage of the one or more stages of the education process.
 22. Thesystem of claim 21, wherein the education related data for the studentincludes information, skills or competences possessed by a teacher ofthe student.
 23. The system of claim 21, wherein the education relateddata for the student includes lesson planning data.
 24. The system ofclaim 21, wherein the education related data for the student includesattendance data for the student.
 25. The system of claim 21, wherein theeducation related data for the student includes at least one of dataindicative of student engagement or data indicative of student behavior.26. The system of claim 21, wherein the education related data for thestudent includes data indicative of remedial or enrichment activitiesperformed by the student.
 27. The system of claim 21, wherein theeducation related data for the student includes at least one ofinformation, skills, competences or learning outcome which the studentis expected to master according to a mastery level.
 28. The system ofclaim 21, wherein computer code instructions, which when executed by theone or more processors, cause the one or more processors to: simulatethe machine learning model to measure an effectiveness of one or moreassessment elements in assessing the amount of information gain for thestudent.
 29. The system of claim 21, wherein the training data and thetest data include at least one of: Program for International StudentAssessment (PISA) data; Trends in International Mathematics and ScienceStudy (TIMSS) data; National Assessment Program (NAP) data Progress inInternational Reading Literacy Study (PIRLS) data; Smarter BalancedAssessment Consortium data; student information data; class or gradeinformation data; resources information data; educational applicationsinformation data; school facilities information data; data related toqualifications, skills, professional development or paygrades ofprincipals, teachers, or staff associated with one or more schools; orbudgeting and expenditure information data for one or more schools. 30.The system of claim 21, wherein the plurality of models includes atleast one of: a Bayesian network model; a directed acyclic graph model;a neural network model; or a convolutional neural network model.
 31. Amethod comprising: obtaining, by one or more computing devices, trainingdata and test data related to student performance for a first pluralityof students, the training data and the test data associated with aplurality of variables having potential influence on performance of aneducation process; generating, by the one or more computing devices,using the training data, a machine learning model to simulate theperformance of the education process with respect to information gainfor students at one or more stages of a plurality of stages of theeducation process, the machine learning model (i) representing a networkof nodes corresponding to a respective subset of variables of theplurality of variables for simulating the performance of the educationprocess, and (ii) illustrating for each variable of the plurality ofvariables a respective strength of predicting the performance of theeducation process; validating, by the one or more computing devices, themachine learning model using the test data; receiving, by the one ormore computing devices, education related data for a student; andsimulating, by the one or more computing devices, the machine learningmodel using the education related data for the student to predict anamount of information gain for the student at a stage of the one or morestages of the education process.
 32. The method of claim 31, wherein theeducation related data for the student includes information, skills orcompetences possessed by a teacher of the student.
 33. The method ofclaim 31, wherein the education related data for the student includeslesson planning data.
 34. The method of claim 31, wherein the educationrelated data for the student includes attendance data for the student.35. The method of claim 31, wherein the education related data for thestudent includes at least one of: data indicative of student engagement;data indicative of student behavior; or data indicative of remedial orenrichment activities performed by the student.
 36. The method of claim31, wherein the education related data for the student includes at leastone of information, skills, competences or learning outcome which thestudent is expected to master according to a mastery level.
 37. Themethod of claim 31, wherein computer code instructions, which whenexecuted by the one or more processors, cause the one or more processorsto: simulate the machine learning model to measure an effectiveness ofone or more assessment elements in assessing the amount of informationgain for the student.
 38. The method of claim 31, wherein the trainingdata and the test data include at least one of: Program forInternational Student Assessment (PISA) data; Trends in InternationalMathematics and Science Study (TIMSS) data; National Assessment Program(NAP) data Progress in International Reading Literacy Study (PIRLS)data; Smarter Balanced Assessment Consortium data; student informationdata; class or grade information data; resources information data;educational applications information data; school facilities informationdata; data related to qualifications, skills, professional developmentor paygrades of principals, teachers, or staff associated with one ormore schools; or budgeting and expenditure information data for one ormore schools.
 39. The method of claim 31, wherein the plurality ofmodels includes at least one of: a Bayesian network model; a directedacyclic graph model; a neural network model; or a convolutional neuralnetwork model.
 40. A non-transitory computer-readable product includingcomputer code instructions stored thereon, the computer-codeinstructions, when executed by one or more processors, cause the one ormore processors to: obtain training data and test data related tostudent performance for a first plurality of students, the training dataand the test data associated with a plurality of variables havingpotential influence on performance of an education process; generate,using the training data, a machine learning model to simulate theperformance of the education process with respect to information gainfor students at one or more stages of a plurality of stages of theeducation process, the machine learning model (i) representing a networkof nodes corresponding to a respective subset of variables of theplurality of variables for simulating the performance of the educationprocess, and (ii) illustrating for each variable of the plurality ofvariables a respective strength of predicting the performance of theeducation process; validate the machine learning model using the testdata; receive education related data for a student; and simulate themachine learning model using the education related data for the studentto predict an amount of information gain for the student at a stage ofthe one or more stages of the education process.